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I.  RESEARCH  CONDUCTED 

A)  PURE  JUMP  DAMAGE  PROCESSES.  Assume  that  a  device  is  subject  to  damage, 
denoted  by  X.  The  damages  process  is  assumed  to  be  an  increasing  pure  jump 
process.  Such  a  process  can  be  expressed  as  follows: 


Xt  "  X0  +  xt 


where 


X.  =  E  (X  -  X  )  . 

t  _  u  u- 

u<t 


Clearly  X^  describes  the  cumulative  wear  due  to  shocks  occurring  during 
(0,t]  and  Xq  is  the  initial  wear. 

Let  (t,K)  be  the  Letfy  system  for  X.  It  is  known  from  the  work  of 
E.  Benveniste  and  J.  Jacod  that,  for  each  positive  Borel  measurable  function 
f  on  R+  x  R+  with  f(x,x)  =  0  ,  for  all  x  in  E  . 

V  £  f<Xs--Xs)l 
s<t 

*  ExUo  lE+  «Xs_,dy)f(Xs_,y>] 

Cinlar  and  Jacod  show  that  there  exists  a  Poisson  random  measure  on  R  *  R 

9  +  + 

2 

whose  mean  measure  element  at  (s,z)  is  du  dz/z  and  a  deterministic 
function  c  such  that 


>:  f(X  ,X  )  =  /  N(ds ,dz) f (X  ,X  +c (X  , z ) ) 

S-  s  rr,  s-  s-  s- 


u<t 


[0,t]xR+ 


almost  everywhere  for  each  function  f  on  R+  *  R+  of  the  above  type. 

Air;  p-r: 

’ " *  "  '  '  '!  rn  d?i c 

■  -  •  h- r*  v . 


In  particular,  it  follows  that 


X  =  X  +  /  c(X  ,z)N(ds  ,dz)  j : > 

°  [0,t]xR+  KA:r?:iff.T.  •. 

Chlsf,  fe.jt.ni c’i i  Information  Division 
The  above  formula  has  the  following  interpretation: 

t  -►  Xt(w)  jumps  at  s  if  and  only  if  the  Poisson  random  measure  N(w,.) 
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has  an  atom  (s,z)  and  then  the  jump  is  from  the  left  hand  limit  X  (w) 

s- 

to  the  right  hand  limit 

X  =  X  +  c(X  ,z)  . 

S  s—  S“ 

The  function  c(x,z)  represents  the  damage  due  to  a  shock  of  magnitude  z 
occurring  at  a  time  when  the  previous  cumulative  wear  equals  x. 

We  determined  conditions  on  the  function  c  and  the  Letfy  kernel  K 
that  insure  that  the  life  distribution  properties  of  the  threshold  right  tail 
probability  G  are  inherited  as  corresponding  properties  of  the  survival  prob¬ 
ability  F(t)  =  E[G  (X  )  ] .  We  note  that  a  stationary  Levy  shock  and  wear 
process  is  a  special  case  of  the  pure  jump  processes  described  above.  This 
follows  from  Ito’s  representation  of  every  increasing  Letfy  process  as  follows: 

X  =  X  +  /  z  N(ds ,dz)  , 

°  [0,t]xR+ 

where  N  is  a  Poisson  random  measure  with  mean  measure  element  at  (s,dz) 
equals  ds  u(dz)  and  u  is  a  Letfy  measure. 

These  results  have  been  submitted  for  publication. 

B)  OPTIMAL  REPLACEMENT  AND  MAINTENANCE  POLICIES  FOR  DEVICES  SUBJECT  TO 
PURE  JUMP  DAMAGE  PROCESSES. 

Assume  that  a  device  is  subject  to  damage  occurring  randomly  in  time 
according  to  a  pure  jump  process  X.  Let  g(w,t):$>xR+  -*  R  be  a  function  des¬ 
cribing  the  cumulative  net  yield  the  device  produces  till  time  t  .  Suppose 
that  g(w,t)  *  f (Xt (w) )  for  some  f:R+  -►  R  .  The  expected  cumulative  dis¬ 
counted  net  yield  from  observing  the  device  till  time  t  equals  E[d(t)f(Xt)] 
We  give  conditions  on  the  functions  d  and  f  that  ensures  the  existence  of 
a  replacement  policy  that  maximizes  the  expected  cumulative  discounted  net 
yield  and  give  a  closed  form  for  that  replacement  time.  In  the  case  d(t)  » 
e  we  show  that  for  a  suitable  choice  of  the  function  f  the  optimal 


»  * 


«'  I 
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replacement  policy  is  a  control  policy. 

The  following  theorem  follows  from  the  work  of  Ikeda  and  Watanabe  or 
Benveniste  and  Jacod  and  the  martingale  characterization  theorem  for  Poisson 
random  measures  due  to  Jacod.  The  theorem  holds  for  a  larger  class  of  functions 
than  the  one  indicated.  However,  the  statement  given  here  is  sufficient  for 
our  purpose.  First  define 

Af (x)  =  /  [f(x+c(x,z))  -  f (x) ]  . 

R+/tO}  z 

Theorem.  Let  f:R,  -*  R  be  increasing  d:R.  -+  R.  be  differentiable. 

——————  +  T  T 

T 

Then,  for  any  stopping  time  T  for  which  E[/Q  d' (t)f (Xt>dt]  exists  and  is 
finite  we  have  that 

E[d(T)f(XT)]  -  d(0) f (x) 

=  E[/J  d'(t)f(Xt)dt]  +  E[/J  d(t)Af(Xt)dt] 

We  proved  the  following: 

Theorem.  Let  f  be  concave  increasing  and  c(x,z)  be  decreasing  in 

the  first  argument.  Let  d  be  decreasing  and  concave.  Let  F  be  the  class 
of  stopping  times  for  which  E[/q  d ' ( t) f (X^)dt ]  exists  and  is  finite.  Define 
u(t)  =  d’(t)f(Xt)  +  d(t)Af(Xt)  . 

Let 

T*  =  inf{ t:u(t)  <  0} 

Then  T*  is  optimal  in  the  sense  that 

E[d(T*) f (X  . ) ]  =  sup  E[d(T) f (X  ) ]  . 

TlF  1 

Theorem.  Let  f  be  concave  increasing  and  c(x,z)  be  decreasing  in 

v  X  s 

the  first  argument.  Let  d(s)  =  e  ,  >  >.  0  •  Let  F.  be  the  class  of 
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T  “At 

stopping  times  for  which  E[/q  e  f(Xt>dt]  exists  and  is  finite.  Define 
A  =  {x:Af(x)  -  Af(x)  <_  0}  .  Then  the  stopping  time 

T*  =  inf{t:XteA} 

is  optimal  in  the  sense  that 

E[e"'T*f(X  )]  =  sup  E[e"ATf(X  )]  . 

TeF1 

C)  A  PRACTICAL  DEFINITION  OF  STABILITY  FOR  OPTIMAL  STOPPING  TIMES. 

The  results  of  this  work  represent  progress  on  question  1(C)  of  the 
proposal.  This  question  asked  whether  the  two  main  types  of  stability  are 
equivalent.  While  we  do  not  provide  a  complete  answer,  the  result  which  is 
obtained  indicates  that  in  a  practical  sense  they  can  be  considered  to  be 
equivalent . 

Briefly,  a  problem  (X,g)  is  considered  to  be  center  stable,  here  X 
is  a  process  and  g  is  a  reward  function  defined  on  the  state  space  of  X,  if 
a  close  to  optimal  solution  to  the  problem  (X,g)  is  close  to  optimal  for  any 
problem  (x\g^)  which  is  sufficiently  "close"  to  the  problem  (X,g)  .  The 
other  broad  class  of  stability  considered  views  (X,g)  as  "stable"  if  a  close 
to  optimal  solution  to  the  problem  (X^.g^)  is  close  to  optimal  for  (X,g) 

l 

provided  (X  ,g^)  is  "close  enough"  to  (X,g)  . 

We  say  that  the  problem  (X,g)  is  weakly-payof f  stable  at  x  if  for  each 
c  >  0  there  exists  a  positive  integer  nQ  such  that  if  X*  is  "close  enough" 

to  X  then  sup  E^g(X^)  >_  iijg(x)  -  c  (here  n^g  is  the  smallest  excessive 

r<n 

—  o 

majorant  of  g  with  respect  to  X^  .)  The  problem  (X,g)  is  said  to  be  stable 
with  respect  to  bounded  times  if  for  every  c  >  0  and  nQ  c  {1,2,...}  there 
is  a  f,  >  0  and  an  «  >  0  such  that  every  a-optimal  time  t  <  n^  for  (X^,g^) 

whenever  (X*,g^)  is  within  a  of  (X,g)  .  The 


is  c-optimal  for  (X,g) 
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concept  of  center  stable  with  respect  to  bounded  times  is  defined  analagously. 

The  main  result  is  the  following: 

THEOREM  1 .  The  following  are  equivalent: 

a)  (X,g)  is  center  stable. 

b)  (X,g)  is  center  stable  with  respect  to  bounded  times. 

c)  (X,g)  is  weakly-payof f  stable. 

d)  (X,g)  is  weakly-payof f  stable  and  stable  with  respect  to 
bounded  times. 

This  research  intended  to  incorporate  this  result  into  a  rewrite  of  his 
paper  entitled,  "Stability  of  Optimal  Stopping  Problems  for  Markov  Processes". 

D)  STABILITY  OF  OPTIMAL  REPLACEMENT  PROBLEMS. 

The  results  in  this  work  represent  progress  on  one  of  the  main  problems  of 
the  proposal,  namely  that  of  developing  a  stability  theory  for  optimal  replace¬ 
ment  problems  for  wear  processes  which  allows  the  modeling  processes  to  have 
different  lifetimes. 

One  considers  a  nonterminating  wear  process  X,  a  fixed  stopping  time  T 

with  uniformly  bounded  expected  lifetime,  a  family  W(F)  of  stopping  times 

which  are  bounded  above  by  F  and  a  family  S  of  related  nonterminating  processes 

A 

for  all  of  which  F  has  uniformly  bounded  expected  lifetime.  A  metric  is  placed 

on  S  x  N(r)  which  reflects  closeness  of  the  processes  in  S  and  of  the  stopping 
A  A 

times  in  N(r)  . 

A  problem  (X,r,  ,g)  is  considered  to  be  stable  if  a  close  to  optimal  re¬ 
placement  solution  to  the  problem  (X,c,g)  is  close  to  optimal  for  any  sufficient 
close  problem  (X'.c.g^)  and  vice-versa. 

If  the  lifetimes  are  fixed,  we  have  stability.  However,  examples  show 
that  stability  of  replacement  problems  is  quite  sensitive  to  variation  in  the 
lifetimes.  These  results  are  currently  being  written  up  in  a  paper  entitled, 
"Stability  of  Optimal  Replacement  Problems". 


-vfc  ' ."•*  V 
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E)  AN  ITERATION  SCHEME  FOR  OBTAINING  APPROXIMATE  OPTIMAL  REPLACEMENT  POLICIES. 

In  this  work  we  analyze  the  following  iterative  technique.  Let  = 

K°g(Xr) 

- —  where  (X,£,g)  is  a  wear  process  with  lifetime  t,  .  Consider  the 

E°(C) 

problem  of  maximizing  the  criterion  iKb^.i)  =  b^E°(i)  -  E°g(XT)  for  x  <_  • 

If  we  are  interested  in  an  c-optimal  policy,  then  tolerances  a  >  0  and 
d  >  0  are  determined  in  terms  of  X,g,c  and  e  so  that,  if  a  (5-optimal  policy 
Tj  for  the  problem  ^(b^,r)  gives  ip  ^  * T 1  ^  —  a  t'ien  s  is  already 

E°g(X  ) 

1 

e-optimal,  otherwise  take  b.  =  -  and  repeat  the  steps  on  the  criterion 

E°(t1) 

tKb^.x)  =  b^E^x)  -  E°g(X)  .  Under  very  reasonable  assumptions  on  g  (it  must 
be  positive  and  bounded  away  from  zero) ,  it  is  shown  that  this  iterative  method 
will  always  supply  an  t: -optimal  replacement  policy. 

This  method  has  several  advantages  over  those  being  employed  in  the  litera¬ 
ture  where  enough  is  assumed  about  the  process  to  associate  a  contraction  mapping 
with  the  optimal  replacement  problem  and/or  the  function  g  is  so  specialized  as 
to  permit  the  identification  of  a  control  policy  by  inspection  of  an  associated 
formula  employing  the  generator  of  the  semigroup  associated  with  X.  This 
iterative  method  needs  no  such  assumptions  on  g  and  works  even  when  a  true  o- 
optimal  policy  does  not  exist.  The  lack  of  a  o-optimal  policy  would  frustrate 
all  cases  in  the  literature  so  far;  especially  the  contraction  mapping  approach. 
These  results  are  written  up  in  the  paper  entitled,  "An  Iterative  Scheme  for 
Approximating  Optimal  Replacement  Policies". 
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